On the smooth Rényi entropy and variable-length source coding allowing errors
نویسنده
چکیده
In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth Rényi entropy characterizes the optimal exponential moment of the codeword length. Index Terms ε source coding, exponential moment, the smooth Rényi entropy, variable-length source coding
منابع مشابه
Improved Bounds on Lossless Source Coding and Guessing Moments via Rényi Measures
This paper provides upper and lower bounds on the optimal guessing moments of a random variable taking values on a finite set when side information may be available. These moments quantify the number of guesses required for correctly identifying the unknown object and, similarly to Arikan’s bounds, they are expressed in terms of the Arimoto-Rényi conditional entropy. Although Arikan’s bounds ar...
متن کاملCumulant Generating Function of Codeword Lengths in Variable-Length Lossy Compression Allowing Positive Excess Distortion Probability
This paper considers the problem of variable-length lossy source coding. The performance criteria are the excess distortion probability and the cumulant generating function of codeword lengths. We derive a non-asymptotic fundamental limit of the cumulant generating function of codeword lengths allowing positive excess distortion probability. It is shown that the achievability and converse bound...
متن کاملThe Rényi redundancy of generalized Huffman codes
If optimality is measured by average codeword length, Huffman's algorithm gives optimal codes, and the redundancy can be measured as the difference between the average codeword length and Shannon's entropy. If the objective function is replaced by an exponentially weighted average, then a simple modification of Huffman's algorithm gives optimal codes. The redundancy can now be measured as the d...
متن کاملSimple one-shot bounds for various source coding problems using smooth Rényi quantities
We consider the problem of source compression under three different scenarios in the one-shot (nonasymptotic) regime. To be specific, we prove one-shot achievability and converse bounds on the coding rates for distributed source coding, source coding with coded side information available at the decoder and source coding under maximum distortion criterion. The one-shot bounds obtained are in ter...
متن کاملA Preferred Definition of Conditional Rényi Entropy
The Rényi entropy is a generalization of Shannon entropy to a one-parameter family of entropies. Tsallis entropy too is a generalization of Shannon entropy. The measure for Tsallis entropy is non-logarithmic. After the introduction of Shannon entropy , the conditional Shannon entropy was derived and its properties became known. Also, for Tsallis entropy, the conditional entropy was introduced a...
متن کامل